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This report is the results of the second year external 
evaluation of a continuing evaluation study of the 
Responsive Environment Early Education Program (formerly 
the Responsive Environment Program for Spanish American 
Children - REPSAC). This program serves as an educational 
intervention providing direct services to "high risk" (low 
birth weight-less than 5% pounds) 3-, 4-, and 5-year-old 
children, living in the area served by the Clovis Municipal 
Schools, Clovis, New Mexico. In addition, the program 
serves as a base for training selected early childhood and 
kindergarten teachers and aides. 

The major goals for the program are: (1) To prevent 
school failure with an intervention program which includes 
early identification and remediation of developmental 
learning deficiencies and to integrate handicapped children 



into the regular school program; (2) To provide in-service 
training to selected early childhood and kindergarten 
teachers and aides employed by various school districts of 
New Mexico; and (3) To disseminate information concerning 
the program. The design for the external evaluation during 
1976-77 was based upon the program objectives as pertains 
to: student achievement; in-service training; and dissemi- 
nation activities. As concerns student achievement (pre- 
and posttests using standardized tests to measure language 
development in Spanish and English, school readiness, and 
self-concept) 3 program impact was determined by a special 
regression analysis model using four dependent variables 
and twelve independent variables. In-service training and 
dissemination activities were subjectively evaluated using 
site visits, observations, records, and self-reports by 
the staff. 

Major findings included: (1) Students made signifi- 
cant gains in language development in English and school 
readiness; (2) Students made some gains in language develop- 
ment in Spanish; however, the gains were not statistically 
significant; (3) Students indicated a positive and continuous 
growth as concerns self-concept and social development; 
(4) The regression analysis data for the objectives pertaining 
to language development in English indicated that the variable 
making the greatest contribution was IQ, which accounted for 
77% of the variance; and (5) The in-service training provided 
to the 47 teachers and aides was extremely successful and 
effective. Findings of the follow-up study of former students 
in the program are reported in a separate study. 
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PROJECT DESCRIPTIVE AND EVALUATIVE INFORMATION 



This report describes an external evaluation study of the Responsive 
Environment Early Education Program (REEEP) during the 1976-77 school year. 
This report is part of a continuation study by an independent consultant 
and service organization with its direction primarily through various 
faculty niembers of the College of Education, Texas Tech University. 



DESCRIPTION OF THE PROGRAM 

This program is designed to serve as an educational intervention pro- 
viding direct services to approximately 40 high risk 3-, 4-, and 5-year old 
children, including the integration of handicapped children, living in the 
area served by the Clovis Municipal Schools, Clovis, New Mexico. Children 
are considered "high risk" as a result of their low birth weight, 5^2 pounds 
or less, and who will probably have accompanying handicaps as they enter the 
first grade. This program attempts to demonstrate that such an early inter- 
vention can provide such children the experiences necessary to succeed and 
remain in the educational mainstream. 

In addition, the program serves as a base for training selected early 
childhood and kindergarten teachers and aides employed by various school 
districts in New Mexico. 

After completing REEEP (1-3 years depending upon the child's age at 
entry), the children will enter the first grade. A follow-up study is 
being conducted on these students as they enter the mainstream of formal 
education (Grades 1-6). 



Development of the Program 

In developing, the program (formerly the Responsive Environment Program 
For Spanish American Children - REPSAC) has drawn heavily upon various exper- 
imentally developed models in early childhood education including: the New 
Nursery School » Northern Colorado University; the responsive environment 
concept of Omar K. Moore; Project LIFE (Language Instruction to Facilitate 
Education); the Piaget Early Childhood Curriculum, and various parent involve- 
ment programs throughout the nation. In addition, the program has partly 
adapted the Early Prevention of School Failure Model, a nationally validated 
Title III ESEA developer-demonstrator project (Peotone, Illinois). Thus, 
beginning with the school year 1975-76, this program was an adaptation of the 
former Responsive Environment Program for Spanish American Children (REPSAC) 
and the Early Prevention of School Failure Project. 

The rationale for the design and development of REPSAC emanated from 
research which indicated that children with a low birth weight, coupled 
with other factors, generally experience childhood difficulties in the 
cognitive areas of development which can result in subsequent retardation as 



they progress through their formal education. Spanish American children 
with' such a low birth weight coupled with a language different from that 
used in the' American educational setting, have additional handicaps. 
Further, Spanish American children with the foregoing handicaps whose home 
environment often does not include toys, materials, games, and media which 
can enrich their childhood experiences enter the first grade with a notable 
disadvantage in comparison to children with such advantages. 

^Recognizing that approximately 22% of the total school population of 
Clovis, New Mexico were children with a Spanish surname and that approxi- 
mately 39% of children enrolled in special education were of Spanish origin, 
and accepting the premise that a high percentage of "high risk" children 
come from this particular ethnic group, the idea of an early educational 
intervention became a reality in the form of REPSAC. REPSAC officially start- 
ed in September, 1971 with 32 students and operated four years (1971-75) 
serving low birth weight Spanish American children.* Beginning with school 
year 1975-76, the Responsive Environment Early Education Program (REEEP) was 
expanded to serve low birth weight children of all ethnic groups. 



Target Children and Criteria for Selection of Participants 

The target group children of the program are 3-, 4-, and 5-year-old low 
birth weight children v/ho are considered educationally handicapped. Criteria 
used to select children to participate in the program are: 1) Low birth 
weight - SH pounds or less; 2) Health history of child; 3) Level of educa- 
tion of parents; 4) Educational attainment of siblings; 5) Home language - 
Spanish or English; and 6) Income of family. 



Faculty/Staff/Advisory Board 

The faculty/staff of REEEP consists of: the director; one certified 
teacher; two teacher-aides; and one custodian/bus driver. 

In addition to the regular faculty/staff, there is a Professional Ad- 
visory Board. The purpose of the Professional Advisory Board is to provide 
the director with guidance and direction of the activities of the program 
and the development of the various program components. The board consists 
of individuals who can provide expertise in the fields of special education, 
early childhood education, bilingual and bicultural education, educational 
technology, and the responsive environment concept. 



*For references pertaining to evaluation results of REPSAC during 
these years, see Bibliography 4-9. 



Location and Physical Facilities 

The project is located about six blocks southwest of the central business 
area of Clovis, New Mexico and the physical facilities blend into the surround- 
ing buildings and are not discernable from the rest of the community except by 
of the houses. The physical facilities of the program consist 
with an adjoining yard. One house, which is a renovated former 
dwelling, is used as the main teaching faciltiy, and the other 



a sign on one 
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building serves as office and workroom which is a renovated former beauty shop. 



Goals and Objectives 



The following goals and objectives give 
and administration of the program. 



direction to the organization 



Goals 



The major goals of the program are: 



1. To prevent school failure with an intervention program which 
includes early identification and remediation of developmental 
learning deficiencies and to integrate handicapped children into 
the regular school program. 

2. To provide in-service training to selected kindergarten teachers 
and teacher-aides employed by various school districts through- 
out New Mexico, 

3. To disseminate information concerning the program. 
Objectives 

The major objectives of the program are: 

1. Student Achievement (At the end of the school year, the student 
wil 1 be able to: ) 

1.1 Demonstrate language ability in English which is normally 
accepted at their age level. Evidence of achievement will 
be determined from significant gain scores of the Peabody 
Picture Vocabulary Test. (Cognitive) 

1.2 Demonstrate language ability in Spanish which is normally 
expected at their age level. Evidence of achievement will 
be determined from significant gain scores of the Test for 
Auditory Comprehension of Language - Spanish, (Cognitive) 

1.3 Demonstrate school readiness in such areas to include: 
listening ability; visual acuity; and recognition of 
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similarities, differences, and numerical analogies. Evidence 
of achievement will be determined from significant gain scores 
of the Readiness Test for Disadvantaged Children, (Cognitive) 

1.4 Exhibit a positive self-concept and favorable emotional devel- 
opment. Evidence of such behavior will be determined from 
rating from the Developmental Profiles which involve periodic 
teacher evaluations in six areas of the affective domain: aware- 
ness of self, self-confidence, interpersonal comprehension, 
sensitivity to others, effectiveness, and tolerance. Evidence 
of growth/development for each age-level will *be determined from 
individual profile sheets plotted in terms of direction and rate 
of growth/development, (Affective) 

2.1 Upon completion of the various in-service training sessions, 

teachers and aides will be able to employ various aspects of the 
curriculum in their own educational settings as as to meet the 
needs of children with developmental lags and learning deficien- 
cies. This will include incorporating the approaches of Piaget 
and Montessori, using various materials such as the Project LIFE 
materials, and using the responsive environment typing booth. 
Evidence of achievement will be determined by observations and 
written examinations, 

3.1 Information concerning the programs and operation of the pro- 
gram will be disseminated by various means such as: 

a. Progress reports to the LEA central office, school 
board, and local area mews media. ' 

b. Si te-visituation by interested individuals and groups, 
both from in and out-of-state. 

c. Copies of the end-of-year Evaluation Report will be 
disseminated throughout the state including the Educa- 
tional Resources Information Center (ERIC). 



Program Activities 

Activities of the program can be classified as: instructional; in- 
service; dissemination; and outreach. 

Instructional Activities 

The instructional activities of the program are conducted in two-half 
day sessions five days a week. Approximately tv/enty students attend the 
morning session and twenty students attend the afternoon session. The stu- 
dents are transported to and from school by a small bus provided by the 
program. 
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Specific learning activities are planned for the children for each 
three-hour day. These learning activities can be generally classified into 
group activities (story telling, reading, painting, cutting, manipulative 
toys, playground activities, and the lunch period) and individualized or 
small group activities (Piaget-Early Childhood Curriculum, Project LIFE, 
Responsive Typing Booth, and the Peabody Language Development Kit), 

Eating the noon meal with attendant language involvement is developed 
as a learning activity: therefore, all of the students are served a hot 
lunch. The morning group is served prior to leaving school ^ and the after- 
noon group is served immediately upon arrival for the afternoon session, 

In-Service Activities 

The majority of the in-service activities of the program were considered 
as a part of the outreach activities (described later). Other types of in- 
service activities included: attendance at various workshops; enrollment in 
selected graduate/undergraduate courses at Eastern New Mexico University; 
formal and informal sessions with various consultants, including members of 
the external evaluation team; and regularly scheduled faculty meetings. 

Dissemination Activities 

Activities which served as means to disseminate information concerning 
the program included: preparation of various brochures; newspaper releases; 
progress reports to the funding agency, central administration office and 
school board; site visitation by interested groups and individuals; various 
speaking engagements by director and faculty; and copies of the evaluation 
report were distributed throughout the state and nation including the Educa- 
tional Resources Information Center (ERIC - Clearinghouse on Rural Education 
and Smal 1 School s) , 

Outreach Activities 

Beginning with school year 1975-76, the outreach activities component 
was added to REEEP,* The purpose of adding the outreach activities to the 
parent center was to give the program the capability of providing replica- 
tion services to various local education agencies. A unique feature of 
this replication service is the capability of taking the training to the 
replication centers by a specially designed and equipped motor coach. 

Three school districts, all in isolated areas and having a large number 
of target children and within a reasonable distance of the parent center, 
requested replication of all or part of the parent program during the 1975-76 
school year. These school districts were: Fort Sumner, Artesia, and Carls- 
bad, all located in eastern New Mexico, In addition, replication by two 



*The outreach activities of REEEP is funded by the Handicapped Children's 
Early Education Program, Bureau of Education for the Handicapped, U, S, Office 
of Education, Grant No, GOO-75-00079, 
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other school districts had been made completing their third year of operation. 
This replication was made by the Clovis-Portales Bilingual Early Childhood 
Program (ESEA, Title VII) with a training site in both Clovis and Portales, 
New Mexico. 

The outreach activities component is designed to have three major func- 
tions; advisory and tr^^ining, diffusion, and evaluation. 

The evaluation of the outreach activities also includes a follow-up 
study of former REPSAC students as they enter the mainstream of education 
(grades 1-6). The school year 1976-77 was the completion of the third year 
of this follow-up study. 



EVALUATION DESIGN 

The design for the external evaluation during 1976-77 was based upon 
the program objectives as pertains to: student achievement (1.1-1.4); in- 
service training (2.1); and dissemination activities (3.1). Evaluation 
procedures for each of these objectives were as follows: 

Student Achievement 

Evaluation Model 

Objectives pertaining to student achievement were evaluated using the 
model as described in the report, Development of Data Colle^gtion and 
Analysis Procedures (New Mexico Title III, ESEA Advisory Council, May, 1976). 

These data collection and analysis procedures involved the use of a 
special regression analysis model which can assist in determining the impact 
of selected program and personal variables on achievement of the program 
participants. The use of this type of model was necessary because of the^ 
non-availability of a comparison or control group. Also, this model provides 
an estimate concerning how the students would have performed if they had not 
received the treatment. This model can be depicted as: 

Yl = Xi + X2...Xn 

Where: 

- measure of program objective 
' Xp = program and personal variables 

The dependent variables, measures of the objectives pertaining to stu- 
dent achievement, were: 

1. Language development in English as measured by the Peabody Picture 
Vocabulary Test (Dunn) 

2. Language development in Spanish as measured by the Test for Auditory 
Comprehension of Language - Spanish (Carrow) 
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3. School readiness as measured by the Readiness Test for Disadvantaged 
Children (Walker) 

4. Self concept and emotional development as measured by the Develop- 
mental Profiles (Bessell and Palomares) 

A description of the tests is included as Appendix A. 

The rationale for selecting the forementioned instruments v/ere: (1) Each 
instrument has been previously used and determined by the project director, 
teacher, and evaluation team to be appropriate to measure the objectives 
pertaining to student achievement; and (2) Each instrument is standardized 
with appropriate normative data with the exception of the Developmental 
Profiles. 

The independent variables used in this model were: 



1. 


Pre-test scores 




2. 


Time of instruction 




3. 


Age 




4. 


Birth Weight 




5. 


Educational leveTof 


mother 


6. 


Educational level of 


father 


7. 


Sex 




8. 


IQ 




9. 


Family Size 




10. 


Ethnic group 




11. 


Family Status 




12. 


Monthly income 





Procedures and Time-Schedule for Collecting Da ta 

ff 

The participants were tested before and after the treatment (instruc- 
tional activities) and statistical significance vyas determined on the 
difference, (Exception was the Developmental Profiles). 

Data to evaluate the self-concept was collected with the use of the 
Developmental Profiles. This instrument ms completed for each student tv/o 
times during the year by the teacher and teacher aide, 

Dates for administering the pretests were September 7-10, 1976, and the 
posttests were administered February 10-15, 1977.* The Developmental Pro- 
files were completed during September and February. 

Analytical Procedures 

Program impact, as concerns student achievement, was determined by a 
regression analysis utilizing four dependent variables and twelve independent 
variables. The procedure allows for the regression of the independent 



*Posttests were administered approximately 3 months earlier this year 
than in previous years because of a request from the New Mexico State Depart- 
ment of Education (Letter dated Jan. 18, 1977). 
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variables on the dependent variables for the purpose of determining which 
variables have .the strongest influence on the learning outcome. 

Data are presented in terms of means, standard deviations, correla- 
tions, intercorrel ations , multiple correlations, and regression equations. 

On-Going Evaluation 

In addition to the above stated measures (pre and posttest), there 
were on-going subjective evaluation procedures. This consisted mainly of 
site-visits by the evaluator so as to observe and become familiar with the 
daily operation of the program. Such observation was necessary so as to 
become familiar with the setting of the various measures. 

Variables Which Could Affect Student/Project OutcofT^es 

There could have been some variables operating in the program during 
the 1976-77 school year which possibly could affect the student/project 
outcomes. Such variables were: age (especially the 3-year-olds), socio- 
economic background of the students, year in program, handicapping condi- 
tions (single and multiple), and learning environment in the home. Also, 
the early posttesting affected the mean gain scores. 

The external evaluator recognized the possibility of such variables 
and took such into consideration during the evaluation process. 



In-Service Training 

The in-service training was provided to approximately 50 teachers, 
aides, and administrators from various parts of New Mexico was a joint 
effort between this program and the outreach component (replication 
services) which is funded by the Bureau of Education for the Handicapped, 

The major goal of this training is to assist selected teachers and 
aides to acquire certain knowledge and skills so as to be more effective 
while working with young, high risk, handicapped, and vulnerable children 
using the concept and processes of responsive environment. 

The external evaluation (formative and summative) of the outreach 
component was also conducted by B. E. Askins and Associates. A summary of 
the evaluation of the in-service training is as follows: 

Research/Evaluation Questions 

As an element of the evaluation of this training, the director and 
teachers who conducted the training posed several questions they wanted 
answered during/after the training which included: 

1. What kind of rapport exists betv/een personnel at the replication 
centers with the training staff? 



2. Are the teachers being supported in their efforts by their 
administrators? 

3. Do the teachers have adequate availability of professional 
materials in the areas of bilingual and early childhood 
education? 

4. How effective was the teaching of the instructional units 
pertaining to the area of concept development in early 
education? 

5. Are the teachers using the responsive environment concept in 
their teaching including flexible room arrangements and 
learning centers? 

6. Do the teachers have a knowledge of handicapping conditions? 

7. Do the teachers know how to screen children? 

8. How effective were the training workshops conducted at the : 
model center? 

9. Are the teacher-aides effective in their role in the classroom? 
10, Was the motor coach utilized effectively in the training program? 

Procedures to Collect Data 

A variety of measures was used to collect data to evaluate and answer 
the research questions pertaining to the effects of the training. These 
included: evaluation of the training workshops; classroom visits using a 
specifically designed classroom observation form, site-visits with admin- 
istrators, teachers, and aides; self-evaluation reports from the trainees 
and trainers; a review of various types of records maintained by the train- 
ing staff; and a reporting of test scores of student growth from two repli- 
cation sites (Clovis and Portales). 

Evaluation of Workshops . Various in-service training workshops were 
conducted during the 1976-77 school year for selected participants. Each 
workshop consisted of five days of training and was conducted at the model 
center. The workshops were evaluated using the McCallon Workshop Evalua- 
tion System. (See Appendix A for a description of this instrument). 

Classroom Observation . Members of the evaluation team made periodic 
classroom visits of participating teachers (Kindergarten) to observe using 
a specifically designed classroom observation instrument. The form was 
designed to note such things as: availability of bilingual and eatly educa- 
tion materials; learning centers; use of the responsive environment concept 
and evidence of long and short range instructional planning. 
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Site-Visits . Members of the evaluation tean conducted periodic site- 
visits involving administrators, teachers, and aides so as to acquire their 
verbal reaction/responses to the training activities. 

Self-Evaluation Reports . At the end of the year, another facet of 
evaluation was in the form of Self-Evaluation Reports so as to obtain: 

1. Administrators' responses (strengths and weaknesses) to training 
activities . 

2. Teachers' and aides' responses (strengths and weaknesses) to 
training activities. 

3. Trainers; responses (strengths and weaknesses) to training acti- 
vities. 



Dissemination 

Assistance in the evaluation of the objective pertaining to dissemina- 
tion activities (3,1) was provided to the project director. Evaluation of 
this objective was primarily the assessment of the quality and quantity of 
the dissemination materials. 



Summary of Evaluation Activities 

A summary of the evaluation activities for 1976-77 i? depicted on the 
following page. As previously stated, the dates for administering the 
posttests were changed from mid May to mid February, A description of the 
various tests referenced on the summary page can be found in Appendix A. 



1'/ 



EVALUATION PLAN SUMMARY CHART 



District: Clovis, New Mexico 

Project: ResDonsive Environment Early Education Program 
Evaluator: B. E. Askins & Associates 
Date: 1976-77 



OBJECTIVE 



Student 

Achieve- 

menc 



In-servic 
Training 



3. 

Dissemi- 
nation 
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EVALUATION ACTIVITY 



1.1 Measure language 
development in Eng- 
lish on pre and 
posttest basis . 

1.2 Measure language 
development in Span 
ish on pre and 
posttest basis. 

1.3 Measure degree of 
school readiness 

on pre and posttest 
basis . 

1.4 Evaluate self- 
concept and emo- 
tional development 
during 3 nonth' in- 
tervals . 

2.1 Assess effects of 
in-service train- 
ing sessions. 



3.1 Assess the quality 
& quantity of di^- 
semination mater- 
ials. 



SAMPLE OR 
TARGET GROUP 



100% of subject 
group (M=AO) 



100% of subject 
group (N-AO) 



100% of subject 
group (:7=A0) 



L00% of subject 
group (N=A0) 



100% of teachers 
and aides 
(N=50) 



INSTRUMENTATION^ 



TTi'^rmTENT 



PPVT(Form A) 
PPVr(Eorra B) 



TACL(Form A) 
TACL(Form A) 



RTDC(Form A) 
RTDC(For.(\ B) 



DP 



DATElSy 



\^S 

Site-visits 
Writ ten 
exams 



Observations , 
Num.be r of news 
releases . 
Number of 
pro f essionai 
articles. 
Quality of 
training 
f i 1ms . 



^LEGEND 
PPVT- 
TACL- 
RTDC= 



DATA COLLECTION 



RESPONSIBILITY 



Evalua tor 



Eva lua to r 
Evalua to r 



Evalua to r 
Evaluato r 



Teache r /Evalua- 
tor 



Direc to r/Evaiu- 
ator 



Directo r/Evalu- 
ato r 



DATE 



ANALYTICAL 
PROCEDURE 



9/3/76 
5/13/77 



9/3/76 
5/13/77 



9/3/76 
5/13/77 



9/15/76 
2/15/77 
5/15/77 



After 
each 
session 
& end ojf 
year 

End of 
year 



Regression 
analysis 



Regression 
analysis 



Regression 
analysis 



Statistical 
summa ry 



Statistical 
summary 



Narrative 
summary 



Poabody Picture Vocabulary Test (Dunn) 
Test for Auditory Comprehension of Language (Carrow) 
Readiness Test for Di^^advantaged Children (Walker) 
Developmental Profiles (Bessel and Palomarcij;) 



WES= Workshop Evaluation System (McCallon) 
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EVALUATION DATA 

The evaluation design primarily focused upon the program objectives 
as pertains to student achievement (1.1-1.4). The specific objectives and 
description of the evaluation design are presented in previous sections of 
this report. 



Number of Participants and Handicapping Conditions 

Number of Participants 

At the beginning of the school year, there were 40 students enrolled in 
REEEP (23 boys and 17 girls); 9 third-year students; 12 second-year students; 
and 19 initially started the program this year. 

There we^^e 12 students who withdrew during the year, and there were 12 
other students who started the program during the year. 

At the end of the year, there were 40 students enrolled (25 boys and 15 
girls); 7 third-year students; 10 second-year students; and 23 first-year 
students. This data includes 3 sets of twins. 

The pretesting phase included 39 students and the posttesting phase in- 
cluded 27 students. The number of students who were both pre and posttested 
was 27. The reason for N=27 was because of student withdrawals and new 
entries into the program. 

Handicapping Conditions 

Of the students enrolled at the end of the year, the following handicap- 
ping conditions were identified: 9 students were handicapped in general lang- 
uage ability; 2 students had speech handicaps; 4 students had physical handi- 
caps; 3 students had emotional problems; and 10 students had various types 
of general ability (cognitive) handicapping conditions. 



Student Achievement 

Objectives 1.1-1.3 were objectively measured with standardized tests 
using a pre and posttest design. The students were measured at the beginning 
of the school year (September 3, 1976) on the three areas. At the end of the 
year (February 10, 1977 for this year), the students were again measured in 
the same areas. Progress in each area was determined by the amount of gain 
accomplished between the pretest and posttest. In addition, gain scores were 
used to compare performance between first, second, and third-year students. 
Gain scores were also used to compare performance of the 4- and 5-year-olds 
participating in the program. All of differences were statistically treated 
using the t^-test with significance being determined at the .05 level. In 
addition, a regression analysis was conducted to determine the impact of 
selected program and personal variables on posttest performance. Since a 
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control group was impossible with this project, the regression analysis 
technique was also used to make an estimate of performance of REEEP 
students if they had not participated in the program. 

Objective 1.4 pertained to the assessment of self-concept and emotional 
development. The Developmental Profiles were completed by the teacher ^ and 
aide twice during the year, September and February. Change/growth for each 
student was determined by averaging the two ratings for each period and plot- 
ting the results on a profile sheet. Group change was determined by taking 
an average of the individual ratings for each of the twelve areas measured. 

Language Development in English 

^Data analyzed from a pre and posttest analysis indicated that students 
participating in the program made significant gains in language development 
in English. The mean gain was 15.49 (p<.001). This is compared to mean 
gain scores in 1975-76 (MG - 43.40, p<.00. / and in 1974(MG = 46.40, p<,001). 
It is assumed that the early posttesting in 1976-77, approximately 3 months, 
accounts for the lower mean gain in 1976-77. 

When the data were analyzed by age and year in program, posttest per- 
formance was higher for older students and those participating in program 
longer. However, mean gains were higher for younger students. 

Pre and posttest means and standard deviations are presented in Table 1. 
Comparisons by age and year in program are presented in Tables 2 and 3, 

Language Development in Spanish 

Data analyzed from a pre and posttest analysis indicated that students 
participating in the program gained in Spanish language ability although the 
gains were not statistically significant. The mean gain of the participants 
was 18.21. This is compared to mean gain scores in 1975-76 (MG = 25.00, 
p<.001) and in 1974-75 (MG = 31.12, p <.001). It is assumed that the early 
posttesting in 1976-77, approximately 3 months, accounts for the lower mean 
gain scores in 1976-77. 

When the data were analyzed by age and year in program, posttest per- 
formance was higher for older students and those participating in the program 
longer. However, mean gains were higher for younger students, 

Pre and posttest means and standard deviations are presented in Table 1. 
Comparison by age and year in program are presented in Tables 3 and 4. 

School Readiness 

Data analyzed from a pre and posttest analysis indicated that students 
participating in the program made sinnificant gains in school readiness. 
The mean gain score of the students was 17.89 (p< .001). This is compared 
to mean gain scores in 1975-76 (MG = 10.57, p<.001) and in 1974-75 (MG = 
16.13, p<.001). It is interesting to note that the early posttest scores 
in this instance was higher than scores from the two previous years. 
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When data were analyzed by age and year in program, posttest performance 
was higher for older students and those participating in the program longer, 
However, mean gain scores were higher for younger students, 

Comparisons by age and year in program are presented in Tables 2 and 3, 



TABLE 1 

PRE AND POSTTEST PERFORMANCE OF REEEP STUDENTS 



TEST/OBJECTIVE 


N 


MEANS 


MEAN 
GAIN 




s 




t 


PPVT/English 


27 
27 


Pre 28.47 
Post 43.96 


15.49 


13, 
10. 


,48 
62 


4, 


,48* 


TACL/Spanish 


27 
27 


Pre 32.45 
Post 50.66 


18.21 


19, 
22. 


.92 

,73 


1, 


,99NS 


RTDC/Readiness 


27 
27 


Pre 15.22 
Post 33.11 


17.89 


11, 
9, 


,63 
,14 


5, 


,68* 



*(p <.001) 



TABLE 2 

AGE AND TEST PERFORMANCE OF REEEP STUDENTS 



TEST/OBJECTIVE 


AGE 


N 


- MEAN - 
GAIN 


PPVT/English 


4 


12 


15. 


68 


5 


15 


11. 


41 


TACL/Spanish 


4 


12 


20. 


16 




5 


15 


16. 


11 


RTDC/Readiness 


4 


12 


24. 


73 




5 


15 


15. 


81 






TABLE 3 






YEAR IN 


PROGRAM AND 


TEST PERFORMANCE OF REEEP 


STUDENTS 








MEAN 


TEST/OBJECTIVE 


YP 


N 


GAIN 


PPVT/English 


1 


9 


19. 


,41 




2 


10 


15, 


,13 




3 


8 


9. 


,63 


TACL/Spanish 


1 


9 


24, 


,63 




2 


10 


20, 


.18 




3 


8 


16, 


,41 


RTDC/Readiness 


1 


9 


21, 


.73 




2 


10 


18, 


,43 




3 


8 


13, 


,66 
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Self Concept and Emotional Development 

Objective 1.4 pertained to developing a positive concept and favorable 
emotional development. Data were gathered on this objective were obtained from 
subjective ratings at two time intervals by the teacher and aides. Means are 
presented for each scale. As indicated in Figure 1, students in the program 
indicated substantial progress in self-concept and emotional development. 
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AWARENESS 

(1) AWARENESS OF SELF 
2.7 7.3 



MASTERY 

(1) SELF CONFIDENCE 
3.1 6.3 



SOCIAL INTERACTION 

INTERPERSONAL 
(II COMPREHENSION 

1.6 5.9 




6th 12th 18th 24th 30th 36th 
WEEK 



6th 12th 18th 24ih 30lh 36th 
WEEK 



6th 12!h 18th 24th 30th 3Cu^ 
WEEK 



AWARENESS 

(2) sensitivity'to others 
3.8 6.6 



MASTERY 

(2) EFFECTIVENESS 
3.5 6.5 



SOCIAL INTERACTION 

12) TOLERANCE 
3.8 7.0 




6lh 



12th 18ih 24th 
WEEK 



30th 36ih 



6th 



12ih 18fh 24th 
WEEK 



30th 36ih 



6th 12th 18th 24th 30th 
WEEK 
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FIGURE 1 - PERSONAL DEVELOPMENT PROFILE FOR REEEP STUDENTS 
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R egression Analysis 

In addition to analyzing data for each objective, a regression analysis 
was conducted to determine the impact of selected program and personal vari- 
ables on posttest performance. The correlations and intercorrelations are 
presented in Tables 4 and 5. 

The variables which tended to contribute most to posttest performance 
are presented in the following paragraphs. 

Language Development in English , The twelve variables in the analysis 
accounted for 91 per cent of the variance on the posttest. The variable 
making the greatest contribution was IQ accounting for 77 per cent of the 
variance. 

Language Development in Spanish. The twelve variables in the analysis 
accounted for 76 per cent of the posttest variance. The three best predictors 
were pretest score, instructional time (morning or afternoon), and family in- 
come accounting for 66 per cent of the variance. 

School Readiness . The twelve variables in the analysis accounted for 
60 per cent of the variance of posttest performance. The two variables con- 
tributing most to posttest performance were IQ and family size. 

Again, it should be pointed out that posttest data were gathered in 
February, three months before the end of school and the usual posttesting 
period. Three additional months in the program before posttesting would 
undoubtedly made some difference in the outcome of these findings. 
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TABLE 4 



CORRELATIONS BETWEEN PROGRAM AND PERSONAL VARIABLES AND POSTTEST PERFORMANCE 





POST 
TEST 


^1 




X2 X3 X4 


r 

^5 


H 


X7 




Xg 






^12 


PEABODY 
ENGLISH 


.44 




-.0 4 -.0 8 .12 


-.09 


-.12 


-.01 


. 88 


. 30 


-.07 


-. 36 


. 09 


CARROW 
SPANISH 


.59 




-27 .49 -.38 


- . 22 


. 45 


. 0 3 


13 


40 








WALKER 
SCHOOL 
READINESS 


.43 




-.03 -.11 .28 


.01 


-.05 


.18 


.61 


.41 


-.20 


- .28 


.03 






^1 


= Pretest 








X7 = 


Sex 












^2 


- Time of Instruction 








Xg = 


I.Q. 












^3 


= Age 








X9 = 


Family 


Size 










X4 


= Birthweight 








^10 = 


Ethnic 


Status 










^5 


= Mother * s Education 








Xii= 


Family 


Status 










^6 


= Father* s Education 








^12 = 


'Income 
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TABLE 5 



INTERCORRELATIONS BETWEEN PROGRAM AND PERSONAL VARIABLES 





^2 


^3 


X4 


r 

X5 


X5 




^8 


Xg 


^10 


Xll 


Xl2 


Xi 1.00 


.11 


.28 


-.25 


-,24 


-.21 


.02 


,22 


, 51 


, 39 


,07 


. 32 


X2 


1,00 


-.26 


,08 


,04 


.32 


-.25 


.01 


.15 


, 30 


,08 


-.14 


X3 




1.00 


,15 


. 33 


-.60 


.01 


,39 


, 31 


, 22 


, 45 


-.54 


X4 






1,00 


.14 


,06 


,40 


, 31 


,04 


, 35 


,05 


.40 


X5 








1.00 


,60 


,04 


,13 


-.19 


. 21 


,25 


.51 


X6 










1.00 


,03 


,10 


,44 


.24 


-.40 


. 49 


X7 












1.00 

• 


.02 


- .24 


-.17 


. 32 


.10 


Xfl 














1,00 


.18 


. 01 


-.52 


. 34 


X9 
















1.00 


-.17 


. 32 


,10 


xio 


















1,00 


, 10 


, 50 • 



^11 " 1,00 -,47 

1.00 



O 2 
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Estimate of Performance of REEEP Students If They Had Not Participated in 
the Program ^ ^ ^ ' — - ' - 

Considerable data exist to support the notion that gains made by part- 
icipants in any educational program are attributable to a variety of sources 
other than the educational treatment being rendered, ^ The most efficient way 
of controlling for sources of variation other than treatment variation is a 
randomized control group design. The next most efficient is a covariance 
deisgn and the least efficient is a one-group design. 

In situations, such as the evaluation of student achievement of students 
in REEEP, where pre and post data on only one-group are taken, the need exists 
to determine a way of estimating how welT students would have performed if 
they had not received the treatment. In an effort to .estimate performance in 
the absence of treatment, a regression analysis was employed with posttest 
performance in English, Spanish, and School Readiness as dependent variables 
and twelve personal and program variables as independent .variables, These vari- 
ables are identified in Table 4, 

In essence, the procedure involved determining the percentage of posttest 
variation attributable to variables other than treatment, considering the stand- 
ard error as the portion of uncontrolled variation, and considering the differ- 
ence between these two known sources of variation and unity one as an estimate 
of the percentage of variation attributable to treatment. It follows, then, 
that the difference between attribute, extraneous, and error variation and unity 
one provides a rough estimate of the treatment impace; therefore, this index can 
be used to estimate how well participating students might have performed if they 
had not been involved in the treatment. 

Using the results of the three regression analyses, the following informa- 
tion is provided. 

English posttest performance 

Non-treatment variation = 91% 

Error Variance = ^% 

Estimated treatment variation = S% 

School Readiness posttest performance 

Non-treatment variation = 78% 

Error Variance = 4% 

Estimated treatment variation = 15% 



^See Campbell and Stanley, Experimental and Quasi Experimental Designs 
for Research, Chicato: Rand McNally, 1953. 
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Spanish Posttest performance 



Non-treatment variation 
Error variance 

Estimated treatment variation 



= 75% 
= 20% 
= 5% 



From these data, it can be noted that the majority of posttest variance 
on all three of the variables is attributable to non-treatment variation. It 
could be said, then, that if REEEP students had not had the treatment, post- 
test variation would have been basically the same, 

It should be pointed out again that posttest data were gathered in 
February rather than at the end of the school year^ These three additional 
months in the program before posttesting would undoubtedly make some differ- 
ence in the outcome of these three regression analyses, 

Significance of Learner Change 

The statistical test used to determine the significance of the learner 
change v/as the jt-test. For language development in English, the resulting t 
value was 4,48 which was significant at the .001 level. For language develop- 
ment in Spanish, the resulting t_ value was 1,99 which was not significant, 
For school readiness, the resulting _t value was 5,68 which was significant at 
the ,001 level . 

Within the limitations of a one-group, pretest posttest design, the 
following conclusions were drawn, 

1. Students participating in REEEP made significant gains in language 
development in English (objective 1,1); therefore, it was concluded 
that this objective was achieved, 

2. Students participating in REEEP made some gains in language devel- 
opment in Spanish (objective 1,2) but the gains were not significant', 
therefore, it was concluded that this objective was not achieved. 

3. Students participating in REEEP made significant gains in school 
readiness (objective 1,3); therefore, it was concluded that this 
objective was achieved, 

4. Students participating in REEEP showed positive and continuous 
growth in self-concept and emotional development (objective 1,4); 
therefore, it was concluded that this objective was achieved, 

The regression analysis yielded information relating to the impact of 
selected variables accounting for the majority of posttest variance. These 
were: IQ for language development in English; pretest score, instructional 
time, and family income for language development in Spanish, and IQ and family 
size for school readiness. 
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In-Service Training 

Objective 2.1 pertained to the in-service training provided to approx- 
imately 50 teachers, aides, and administrators from various parts of New 
Mexico. This training was a joint effort between this program and the 
outreach component (replication services) which 1s funded by the Bureau of 
Education for the Handicapped, The REEEP facilities was the location of 
the training, and the REEEP teacher and aides served as demonstration 
teachers in conjunction with the outreach staff. 

Six 5-day in-service training workshops, Early Childhood Education, 
were conducted with approximately 8 trainees in each workshop. Dates of 
the workshops were Oct. 11-15, Nov. 8-12, Dec. 6-10 (1976), January 24-28, 
Feb. 14-18, and March 7-11 (1977). 

The purpose of each workshop was to provide training on the practical 
problems of organizing, planning, and implementing an effective preprimary 
program for children in New Mexico, Special emphasis was given to techniques 
for working with high risk, developmentally slow or disabled children. Act- 
ivities included orientation, guided observation, participation, seminars, 
demonstrations, independent study, and techniques in the use of learning 
materials. 

A total of 47 trainees attended the six workshops, and eight more are 
scheduled during the final workshop April 11-15, 1977. 

Using the McCallon Workshop Evaluation System , the mean ratings on the 
seven criteria/dimensions of the Workshop Evaluation Scale for all of the 
workshops were well above average (6.62 - 6.74 on a 7.00 scale), and the mean 
rating of the Overall Effectiveness of the workshops was 6.72. The mean 
ratings of the seven evaluative criteria, standard scores, and percentile 
ranks are presented in Table 6. The standard scores and percentile ranks were 
computed from the mean ratings using the McCal Ion Manual , 

TABLE 6 



SUMMARY EVALUATION DATA FOR SIX WORKSHOPS* 



Criteria 


Mean 


Standard 
Score 


Percenti 1 e 
Rank 


Organization 


6.74 


118 


90 


Objectives 


6.62 


114 


86 


Consul tants 


6.70 


112 


83 


Ideas/Activities 


6.66 


116 


86 


Sccpe 


6.72 


122 


95 


Benefit 


6.66 


120 


90 


Overall Effectiveness 


6.72 


114 


90 



*N = 47 
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In addition, the narrative statements concerning the observations of 
the workshops were very positive with very few negative comments. 

Based upon the results of these findings, it is obvious that the in- 
service training was extremely successful and effective. Participant reac- 
tion to the workshop goals/objectives was highly favorable and the vast 
majority of the personal comments indicated a high degree of satisfaction. 
In short, the training was effective and was conducted by qual if ied^^afid ex- 
tremely dedicated personnel to an appreciative group of people; ,t]iefef ore, 
it was concluded that objective 2.1 was achieved. 

Dissemination Activities 

Objective 3,1 pertained to the quantity and quality of dissemination 
of information concerning the project. Information concerning the project 
during the 1976-77 school year was disseminated as described in the follow- 
ing paragraphs. 

Progress reports were made available to the central administration 
office, school board, local area news media including nearby Cannon Air 
Force Base, the State Department of Education, and the U.S, Office of 
Education, 

The project did accept the invitation and applied for a Title III 
validation (U,S. Office of Education) in February, 1977. If approved, 
the project will be nationally identified as a demonstration school for 
early childhood training. 

Presentations concerning' the project were made by the project direc- 
tor at Salt Lake City, Utah (Jan 77); San Diego, California (Feb 77); 
and at the annual meeting of the New Mexico Council for Exceptional Child- 
ren, Eastern New Mexico University (Mar 77). 

Papers were presented by the evaluator concerning the project at 
the annual meeting of the American Educational Research Association, New 
York City (Apr 77) and at the 1976 study conference of the Texas Associa- 
tion for Childhood Education (Nov 76). 

Site visitations were made by many professional individuals and 
groups as well as parents. This also included visitations by students from 
the local high school (Home Economics Dept) as well as from the Employee- 
a-Teen Project (federal project in Clovis). 

Copies of the 1975-76 end-of-year evaluation report v/ere disseminated 
throughout the states of New Mexico and Texas. This report was accepted 
into the network of the Educational Resources Information Center (ERIC) in 
the Clearinghouse of Rural Education and Small Schools (See Bibliography 
9). Also, it should be noted the end-of-year evaluation study of the project 
for each year since 1971-72 is now in the ERIC system (4, 5, 6„ 7, and 9) 
and that a total of 6 articles pertaining to the project now appears in 
various professional journals (See Bibliography). 
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In addition to the above, much information concerning the project 
was disseminated by the outreach activities component as well as during the 
6 in-service training workshops conducted during 1976-77. 

Based on these data, observations, and interviews with various 
school personnel, it was concluded that the quality and quantity of 
disseminated materials were more than adequate. Therefore, it was concluded 
that the objective pertaining to the dissemination of Information was 
achieved. 

SUMMARY OF EVALUATIVE INFORMATION, CONCLUSIONS, AND RECOMMENDATIONS 

This project was designed to serve as an educational intervention 
providing direct services to approximately 40 high risk (low birth weight- 
less than 5^2 pounds) 3-, 4-, 5-year-old children, including the integra- 
tion of handicapped children, living in the area served by the Clovis 
Municipal Schools, In addition, the project served as a base for training 
selected early childhood and kindergarten teachers and aides employed by 
various school districts of New Mexico. 

The evaluation design for 1976-77 focused primarily upon the program 
objectives as pertains to student achievement. Also, the objectives 
pertaining to in-service training and dissemination of information were 
assessed . 

Summary of Evaluative Information 
The major findings of this 1976-77 evaluation study were: 

1. Students participating in REEEP made significant gains in language 
development in English (objective 1.1) and school readiness 
(objective 1,3). When data for these two objectives were analyzed 
by age and year in program, posttest performance was higher for 

' the older students and those participating in the program longer; 
however, mean gain scores v/ere higher for the younger students. 

2. Students participating in REEEP made some gains in language 
development in Spanish (objective 1.2); however, the gains were 
not stati sfical ly significant. 

3. Students participating in REEEP showed a positive and continuous 
growth as concerns self-concept and emotional development. 

4. The regression analysis data for the objectives pertaining to 
student achievement indicated that the variables which tended to 
contribute most to posttest performances were: 

A. Language Development in English 

The variable making the greatest contribution was IQ accounting 
for 11% of the variance. 

B . Languag e Development in Spanish 

The three best predictors were pretest scores, instructional 
. time (morning or afternoon), and family income accounting 

for 66% of the variance. 
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C. School Readiness 

The two variables contributing most to posttest performance 
were IQ and family size. 

The regression analysis data indicated that the majority of post- 
test variance on each of the three objectives is attributable 
to non-treatment variation; therefore if the REEEP students had 
not had the treatment, posttest variation would have been basically 
the same. 

5. The external evaluation team found the REEEP students to be 
extremely friendly and cooperative, willing to try various 
books without fear of failure, and an unusually long attention 
span for this age and type of children. 

6. The in-service training provided to the 47 teachers and aides 
was found to be extremely successful and effective. 

7. The quantity and quality of the dissemination activities of the 
project were more than adequate. 

Concl usions 

Based upon the findings of this study, the major conclusions were: 

1. The objectives pertaining to language development in English 
(1.1) and school readiness (objective 1.3) were satisfactorily 
achieved; however, the objective pertaining to language develop- 
ment in Spanish v/as not achieved. 

2. The objectives pertaining to in-service training (objective 2.1) 
and dissemination of project information (objective 3.1) were 
satisfactorily achieved. 

3. The program is in an active and positive process of accomplishing 
the long range goals. 

4. The project has the organization, curriculum, facilities, and a 
qualified and dedicated faculty/staff to provide the needed 
educational experiences for the target children; therefore, it 
was concluded that REEEP is serving as an effective educational 
intervention for the specified target children. 

Recommendations 

Based upon the findings and conclusions of this study, the following 
suggestions or recommendations v/ere made: 

1. That the project continue to develop and serve as an early 
childhood intervention program and as a demonstration and 
repl i cation model . 

2. That more emphasis or time be placed upon the instructional 
activities designed for language development in Spanish. 
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That the evaluation design for the next school year include the 
regression analysis. 

That posttesting be accomplished only at the end of the school 
year. 

That the follow-up study of former REPSAC/REEEP students be 
made a part of the external program evaluation of REEEP beginning 
with the 1977-78 school year. 
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APPENDIX A 
DESCRIPTION OF INSTRUMENTS 



A brief non-technical description of each of the test instruments 
is listed in the following paragraphs. Personnel interested in more detail 
concerning the tests are invited to consult technical data provided by the 
publishers of the tests or refer to the Mental Measurements Yearbook , 
Buros, editor. 



Language Development 

English 

The Peabody Picture Vocabulary Test (Dunn) is designed to provide 
an estimate of a subject's "verbal intelligence" through measuring his 
hearing vocabulary. The test also has wide utility as a clinical tool. 
Besides being effective with average subjects, it has special value with 
certain other groups. Since subjects are not required to read and the 
responses can be non-oral, the test is especially fair to non-readers and 
remedial reading cases. With the drawings free of fine detail 'and figure- 
ground problems, the test is apparently appropriate ^for at least some 
perceptually impaired persons. According to the Test Manual , the ,scale is 
appropriate for subjects l\ - 18 years who are able to hear words, see the 
drawings, and have the facility to indicate "yes" and "no" in a manner 
which communicates. 

This standardized test is published by American Guidance Services, 
Inc., Circle Pines, Minnesota. 

Spanish 

The Test for Auditory Comprehension of Language (Carrow) measures the 
child's understanding of the Spanish language structure. The test is com- 
posed of 101 plates of pictorial referents. The child responds to each of 
the examiner's oral stimuli by pointing to one of three line drawings. Re- 
sponses are recorded on a separate scoring/analysis form. The test j's 
designed for individual administration by speech and testing specialists. 
Test results can also be used to diagnose the language competence of bilin- 
gual and mentally retarded children as well as those with hearing, articula- 
tion, or language disorders. 

. This standardized test is published by Learning Concepts, Inc., 
Austin, Texas. 



School Readiness 

The Readiness Test for Disadvantaged Pre-School Children (Walker) 
was adapted from the final report of a project conducted by Dr. Wanda 
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Walker, Northwest Missouri State College, and supported by the Office of 
Education. The test consists of multiple-choice items based on pictures 
and symbols which do not require reading ability and are designed to test 
a child's listening ability; visual ability; and his recognition of simi- 
larities, differences, numerical analogies, and missing parts. 

This standardized test is available from ERIC Reproduction Service 
(ED 047 168), Bethesda, Maryland. 



Self Concept and Personality Development 

The Developmental Profiles (Bessell and Palomares) is a subjective 
evaluation of children's behavior under a variety of circumstances. 
These rating scales are prepared periodically jointly by two teachers. 
The teachers make ratings on a printed form according to six affective 
areas: awareness of self; self-confidence; interpersonal comprehension; 
sensitivity to others; effectiveness; and tolerance. Because of the in- 
herently subjective nature of these profiles, there is no objective scale 
of accomplishment or standard in terms of age-achievement scores. The 
profiles can provide a source of insight and understanding of emotional 
and personality development. 

This instrument is published by the Human Development Training Insti- 
tute, El Cajon, California. 



Workshop Evaluation 

The Workshop Evaluation System (McCallon) provides a scientific ap- 
proach to gathering and using participant feedback in the evaluation of 
conferences, workshops, conventions, and in-service training programs. 

Participant responses are gathered on seven dimensions - organization, 
objective, work of the presenter, ideas and activities, scope, benefit, 
and overall effectiveness. The Workshop Evaluation System is unique in that 
it provides normative data collected from over 40,000 workshop participants, 
participant feedback scores are compared against the norms to ensure reli- 
able analysis and interpretation of workshop effectiveness. 

This instrument is published by Learning Concepts, Inc., Austin, Texas. 



